Skip to content

fix: harden synced issue format guard - #2969

Merged
stranske merged 3 commits into
mainfrom
automation/issue-format-review-repair
Aug 7, 2026
Merged

fix: harden synced issue format guard#2969
stranske merged 3 commits into
mainfrom
automation/issue-format-review-repair

Conversation

@stranske

@stranske stranske commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Repairs source-owned review debt in the issue-format guard and its synced template copy.

  • do not let language-qualified fence markers close a code fence
  • accept required headings with trailing qualifiers
  • clear stale agents:formatted for any invalid non-exempt issue change without failing on transient label API errors
  • make the consumer-facing test reference explicitly upstream

Validation: python -m pytest -q tests/scripts/test_issue_format.py; python scripts/sync_tool_versions.py --check; git diff --check.

Summary by CodeRabbit

  • Bug Fixes

    • Improved issue formatting validation for fenced code blocks and headings with parenthesized qualifiers.
    • Prevented nested or trailing-content fence markers from incorrectly closing code blocks.
    • Ensured formatting labels are cleared after any invalid issue validation, with non-blocking warnings if removal fails.
    • Updated validation messages to accurately describe invalid issue content.
  • Tests

    • Added regression coverage for qualified section headings and nested code fences.

Copilot AI lite review requested due to automatic review settings August 7, 2026 09:32
@cursor

cursor Bot commented Aug 7, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@stranske
stranske temporarily deployed to agent-high-privilege August 7, 2026 09:32 — with GitHub Actions Inactive
@stranske-keepalive

Copy link
Copy Markdown
Contributor

Workflow source needed

PR #2969 needs either a linked GitHub issue or one valid non-issue Workflow Source before PR metadata automation can manage it safely.

Please do one of:

  • Add <!-- meta:issue:123 --> or a normal Closes #123 / Related to #123 line.
  • Check one Workflow Source option in the PR body.
  • Add a hidden marker such as <!-- workflow-source:local_request -->, <!-- workflow-source:manual_remote -->, <!-- workflow-source:review_followup -->, <!-- workflow-source:sync_campaign -->, or <!-- workflow-source:dependabot -->.
  • Add a workflow source label such as workflow:source-direct-pr, workflow:source-local-request, workflow:source-review-followup, workflow:source-sync, or workflow:no-automation.

Once a valid source is present, this warning will not be reposted.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 53 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7122472b-32e5-4725-873d-c41d985c5041

📥 Commits

Reviewing files that changed from the base of the PR and between 8c5f1b5 and 7917e5b.

📒 Files selected for processing (6)
  • .github/scripts/issue_format.py
  • .github/sync-manifest.yml
  • .github/workflows/agents-issue-format-guard.yml
  • docs/ci/WORKFLOWS.md
  • templates/consumer-repo/.github/scripts/issue_format.py
  • templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml
📝 Walkthrough

Walkthrough

The issue validator now requires standalone closing fences and accepts qualified section headings. Matching consumer templates and regression tests were updated. Guard workflows now clear stale agents:formatted labels for all non-exempt invalid issue bodies.

Changes

Issue format guard

Layer / File(s) Summary
Validator parsing and regression coverage
.github/scripts/issue_format.py, templates/consumer-repo/.github/scripts/issue_format.py, tests/scripts/test_issue_format.py
Fence closing requires a matching marker followed only by whitespace. Required headings may include parenthesized qualifiers. Tests cover nested language-marked fences and qualified headings.
Invalid-body label invalidation
.github/workflows/agents-issue-format-guard.yml, templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml
The workflows clear agents:formatted after any non-exempt invalid validation. Label-removal failures produce warnings, and the status message refers to invalid issue bodies.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested labels: autofix:patch, verify:compare

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: hardening the synced issue-format guard.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch automation/issue-format-review-repair

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8c5f1b5041

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/agents-issue-format-guard.yml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/agents-issue-format-guard.yml:
- Around line 89-91: Update the invalid-issue label-removal branch in
.github/workflows/agents-issue-format-guard.yml lines 89-91 and
templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml lines
89-91 so the cleared message is emitted only when gh issue edit succeeds; move
the warning into the failure branch and avoid reporting success after an API
failure.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: cad5fbe3-df5d-471f-b2fe-57e6f99cfed1

📥 Commits

Reviewing files that changed from the base of the PR and between a05848a and 8c5f1b5.

📒 Files selected for processing (5)
  • .github/scripts/issue_format.py
  • .github/workflows/agents-issue-format-guard.yml
  • templates/consumer-repo/.github/scripts/issue_format.py
  • templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml
  • tests/scripts/test_issue_format.py

Comment thread .github/workflows/agents-issue-format-guard.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the issue-format guard by tightening Markdown fence handling and making required-section heading detection more tolerant, while keeping the consumer template copies in sync with the source.

Changes:

  • Prevent language-qualified fence lines (e.g., ```python) from being treated as code-fence closers when scanning headings.
  • Treat required headings as present when they include a trailing parenthetical qualifier (e.g., ## Tasks (in order)).
  • Make stale agents:formatted label clearing tolerant of label-removal failures (warn instead of failing), in both source and template workflows.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/scripts/test_issue_format.py Adds regression coverage for language-qualified fence markers and qualified required headings.
templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml Mirrors workflow behavior updates for clearing stale agents:formatted without hard-failing on removal errors.
templates/consumer-repo/.github/scripts/issue_format.py Mirrors validator logic changes for fence closing and qualified required headings.
.github/workflows/agents-issue-format-guard.yml Updates stale-label invalidation to run on any invalid non-exempt check and warns on removal failures.
.github/scripts/issue_format.py Updates heading scan to only close fences on “marker-only” lines and matches required headings with parenthetical qualifiers.

Comment thread .github/workflows/agents-issue-format-guard.yml Outdated
Comment thread templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml Outdated
Replace percent-format fence regex with rf-specifiers and align WORKFLOWS/sync-manifest with full agents:formatted clear paths.

Co-authored-by: Cursor <cursoragent@cursor.com>
@stranske

stranske commented Aug 7, 2026

Copy link
Copy Markdown
Owner Author

Closer recovery (cursor) — head 38ccd97e

Addressed Gate/Selftest lint-ruff UP031 on both synced issue_format.py copies (percent-format fence regex → rf format specs) and the Codex P2 docs thread: docs/ci/WORKFLOWS.md + .github/sync-manifest.yml now state that any invalid non-exempt issue change on open/edit/reopen/eligible hold-exemption label events/manual dispatch clears stale agents:formatted.

Local validation: ruff check on both scripts clean; pytest -q tests/scripts/test_issue_format.py → 15 passed.

Not merging this round: mandatory post-push review window + fresh required checks must complete on head 38ccd97e.

Avoid claiming the stale completion marker was cleared when gh issue edit fails.

Co-authored-by: Cursor <cursoragent@cursor.com>
@stranske-keepalive

stranske-keepalive Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Automated Status Summary

Head SHA: 4bf6d37
Latest Runs: ⏳ pending — Gate
Required contexts: summary
Required: core tests (3.12): ⏳ pending, core tests (3.13): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job Result Logs
(no jobs reported) ⏳ pending

Coverage Overview

  • Coverage history entries: 1

Coverage Trend

Metric Value
Current 76.22%
Baseline 85.00%
Delta -8.78%
Minimum 70.00%
Status ✅ Pass

Top Coverage Hotspots (lowest coverage)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/validate_template_sync.py 0.0% 81
scripts/langchain/topic_splitter.py 19.1% 57
tools/codex_log_analyzer.py 19.6% 140
scripts/repo_review_round2_runner.py 25.1% 344
scripts/prune_agent_stubs.py 39.7% 26
scripts/repo_review_round1_runner.py 40.7% 133
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
tools/discover_model_catalog.py 44.8% 55
scripts/repo_review_backlog_scan.py 45.3% 116
scripts/repo_review_body_writer.py 46.5% 86
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58

Low Coverage Files (<50.0%)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/validate_template_sync.py 0.0% 81
scripts/langchain/topic_splitter.py 19.1% 57
tools/codex_log_analyzer.py 19.6% 140
scripts/repo_review_round2_runner.py 25.1% 344
scripts/prune_agent_stubs.py 39.7% 26
scripts/repo_review_round1_runner.py 40.7% 133
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
tools/discover_model_catalog.py 44.8% 55
scripts/repo_review_backlog_scan.py 45.3% 116
scripts/repo_review_body_writer.py 46.5% 86
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58

Updated automatically; will refresh on subsequent CI/Docker completions.


Keepalive checklist

Scope

No scope information available

Tasks

  • No tasks defined

Acceptance criteria

  • No acceptance criteria defined

@stranske

stranske commented Aug 7, 2026

Copy link
Copy Markdown
Owner Author

Closer same-lane follow-through (cursor) — head 7917e5bf

Second push after review threads landed on the first recovery head:

  • gate success-log accuracy for agents:formatted removal in both source + synced agents-issue-format-guard.yml
  • resolved Codex docs thread + CodeRabbit/Copilot success-log threads (4/4 resolved)

Awaiting mandatory ≥7m post-push review window and green required checks on 7917e5bf before merge.

@stranske
stranske temporarily deployed to agent-high-privilege August 7, 2026 09:39 — with GitHub Actions Inactive
@stranske
stranske merged commit 202f75f into main Aug 7, 2026
54 checks passed
@stranske
stranske deleted the automation/issue-format-review-repair branch August 7, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants